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1 Introduction 


Differences in earnings across graduates of different disciplines rival, and in some cases 
exceed, the difference in earnings between college and high school graduates (Arcidiacono 2004; 
Altonji, Blom, and Meghir 2012; Altonji, Kahn, and Speer 2016; Altonji, Arcidiacono, and 
Maurel 2016). As in many STEM (Science, Technology, Engineering, Mathematics) fields, an 
Economics degree offers high future salaries, but the share of graduates from racial and ethnic 
minorities is low, contributing to workplace inequality (Dynan and Rouse 1997; Siegfried 2018; 
Altonji, Blom, and Meghir 2012).! The well-documented underrepresentation of Black, 
Hispanic and Native American people in the Economics profession (Bayer and Rouse 2016; 
Bayer, Hoover, and Washington 2020) may also have consequences for the questions studied 
by economists and corresponding policy recommendations (May, McGarvey, and Whaples 
2014). The lack of diversity in the field begins with undergraduate Economics majors. 

One channel that impacts major choice is information. In choosing a college major, students 
form beliefs about the earnings and utility they expect to receive from potential majors 
(Stinebrickner and Stinebrickner 2013; Zafar 2013), and revise these beliefs in response to new 
information (Wiswall and Zafar 2014; 2015). Sharing information is a “Best Practice” 


recommended by the American Economic Association (AEA) to “correct gender and 


racial/ethnic disparities in knowledge about economics” (Bayer et al. 2019). If students enter 
college with limited information about what economists study, and if either knowledge or 
preferences for career paths vary across student groups, then informing students about the 
true variety of topics in Economics could increase diversity. The AEA suggests using email as 
a method to address knowledge deficits and misconceptions held by students, noting the 
effectiveness of such interventions: “When faculty proactively offer information about the 


breadth of the field of economics, more students from underrepresented groups study 


! In 2014, underrepresented minorities earned 15% of bachelor’s degrees in Economics, compared to 
22% of all bacherlor’s degrees and 20% of STEM (Science Technology, Engineering, and Mathematics) 
degrees (Bayer and Rouse 2016). 
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economics” (Bayer et al. 2019). This paper tests that claim, focusing on underrepresented 
minorities (URM) and first generation students (i.e., students whose parents did not complete 
a four-year degree) as the underrepresented groups. We provide evidence in a different context 
from previous work, and vary the type of information provided. 

We designed a randomized control trial to test whether students respond to messages about 
majoring in Economics. The experiment included more than 2,200 students enrolled in 
Economics Principles courses at Oregon State University. We randomly assigned students to 
receive messages emphasizing the rewarding careers or financial returns associated with the 


Economics major. The rewarding careers message took two forms, a video produced for wide 


distribution by the AEA (henceforth, the “AEA video”) or a local version featuring current 


and recent Economics students at the university (“OSU video”). The AEA video is used by 
many departments attempting to attract underrepresented students by correcting information 
gaps, and the OSU video allows us to test for role model effects of receiving similar information 
from peers. The financial returns message (“earnings information”) contrasted salaries for 
Economics graduates with those from other majors. We compare these groups to students 
receiving no email (“control”) and to a group receiving a message with basic information about 
the major. All message content aligns with recommendations to promote diversity in 
Economics (e.g., Bayer, Hoover, and Washington 2020). This paper is a companion to Pugatch 
and Schroeder (2021), which analyzed the effects of the same experiment on the gender 
composition of Economics majors. 

These email messages increased the probability that first generation and underrepresented 
minority students went on to major in Economics by five percentage points. This effect size 
was sufficient to reverse the gap in Economics majors between first generation/URM students 
and students not in these groups. Effect sizes were larger and more precise for better- 
performing students and first generation students. The outcome is drawn from administrative 
data collected in the academic year following treatment, meaning the effects represent a 


durable change in revealed preference. 


We use our results to conduct a thought experiment. How would the proportion of first 
generation/URM Economics majors change if the intervention became departmental policy? 
In this scenario, the first generation/URM proportion would double from 0.18 to 0.36. For 
students earning a B- or better, the implied increase is from 0.14 to 0.35, or 151%. 

We contribute to the burgeoning literature on promoting interest in undergraduate 
Economics among groups underrepresented in the field. The large scale and negligible marginal 
cost of our experiment—the intervention consisted of a single email—help to understand the 
frontier of informational nudges to promote undergraduate Economics. We complement Bayer, 
Bhanot, and Lozano (2019), who test similar messages among incoming students at liberal arts 
colleges, by studying introductory Economics students at a less-selective public university. 
Both studies find positive effects of an information intervention on interest in Economics 
among first generation and URM students. Whereas in Bayer, Bhanot, and Lozano (2019) 
these effects faded after an academic year, in our case the outcomes were measured in the 
academic year following the experiment, suggesting durable behavior change. Our results are 
also consistent with findings that perceptions of a college major can influence student choices 
(Haggag et al. 2021). 

Related work looks at female participation in Economics. In our experiment, only male 
students (unconditional on first generation/URM status) majored in Economics at higher rates 
in response to the messages. As a result, the same counterfactual thought experiment we 
conduct in this paper would dramatically decrease the proportion of female Economics majors 
in our sample (Pugatch and Schroeder 2021). Experimental evidence from the Undergraduate 
Women in Economics (UWE) challenge (Avilova and Goldin 2018) suggests that deeper 
engagement may be required to increase female interest in Economics. Li (2018) finds that a 
package of information, nudges, and mentoring increased the probability of majoring in 
Economics for female students whose grades were above the median. Porter and Serra (2019) 
find that female students were significantly more likely to major in Economics when a female 


role model visited their Principles class. Whereas these interventions included personal 
interaction with students—via class presentations, mentoring, or role model visits—our study 


and Bayer, Bhanot, and Lozano (2019) included only impersonal, electronic communication. 
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Although the Bayer, Bhanot, and Lozano (2019) experiment increased diversity within their 
full sample of underrepresented students, they failed to find statistically significant results in 
the female subsamples. Similarly, other RCTs intended to increase interest in Economics using 
nudges without personal interaction had only modest to null results among female students 
(Antman, Flores, and Skoy 2020; Halim, Powers, and Thornton 2021). 

Our work therefore helps to draw a more nuanced picture of the AEA’s recommendation 
to share information. Gaps in information or perceptions may be unevenly distributed among 
different underrepresented groups in Economics, leading to different responses to informational 
interventions. Larger information gaps may allow arms-length information interventions to 


increase interest in Economics among first generation and URM students. 


2 Research Design 


2.1 Context 


The study occurred at Oregon State University (OSU), the largest university in the state, 
with 31,000 students. In the academic year of the study, 2018-2019, first generation students 
made up 23% of the student body. Underrepresented minority (URM) students were 12% of 
students. We follow OSU’s definition of URM as American Indian/Alaska Native, Black, 
Hispanic, and Native Hawaiian/Pacific Islander students of U.S. origin. 

The academic year at OSU consists of three 10-week terms. The Economics Principles 
sequence includes two courses, Introduction to Microeconomics and Introduction to 
Macroeconomics. The Economics major is relatively small, with fewer than 100 degree 
recipients in the year of the study. Nonetheless, the Principles classes are popular, fulfilling 
course requirements for 40 other majors and 15 minors. Within our sample of Principles 
students, the most popular majors at baseline are business (49%) and engineering (26%). The 
sample includes eight sections of Introduction to Microeconomics and five sections of 
Introduction to Macroeconomics, none of which were taught by an underrepresented minority. 
Most Principles students take one course or the other. Students who take both courses may 


take them in either order, and occasionally take them simultaneously. 
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Admitted students to Oregon State are assigned the major listed as their preference when 
applying. Undecided students may choose “University Exploratory Studies,” which assigns 
them an academic advisor and other services before they choose a disciplinary major. In our 
sample, 8% of students are in this or similar exploratory programs at baseline. Students who 
want to switch to Economics from another major do not need departmental approval, but 
must meet with an academic advisor. These institutional features may lead to persistence in 
major choice compared to institutions where “undecided” is the default major for entering 


students. 


2.2 Experiment 


We invited all students registered in Economics Principles courses on OSU’s main Corvallis 
campus to participate in the study. Participants were randomly assigned to the following 


groups: 


1. Control no encouragement message 

2. Basic information: encouragement message based on description of Economics 
major on departmental website. 

3. Earnings information: basic information, plus information on earnings of Economics 
graduates one and fifteen years after graduation. 

4. AEA video: basic information, plus link to American Economic Association video 
“A career in Economics...it’s much more than you think.” 

5. OSU video: basic information, plus link to video testimonials by current Economics 


students and alumni of Oregon State University. 


All treatments align with recommendations to promote diversity in Economics. Sharing 
information about the major is an AEA Best Practice for working with students. Moreover, 
the content of the earnings information, AEA video, and OSU video treatments matches 


perceived information gaps among groups underrepresented in Economics. Respondents to a 


survey of underrepresented minorities in Economics “wished they had more information on 
“lwlhat you can do with an economics degree...Respondents also wished they had known more 
about economic research and what academic economists do outside of the classroom. They 
wanted salary information too” (Bayer, Hoover, and Washington 2020, p. 201-202). 

Treated students were sent one message, in Week 8 of the 10-week course, from the email 


account of the student's instructor.? All emails had an identical subject line, “ECON [201/202]: 


Consider majoring in Economics!” Messages appear in Appendix Figure Al. We repeated the 
experiment in each of the three terms (fall, winter, and spring) of the 2018-2019 academic 
year.® 

We randomly assigned treatments at the level of individual students, stratifying by course 
section and class year (freshman/sophomore/other). We assigned each treatment with equal 
probability within strata, though the total number of students in each group differed due to 
uneven strata sizes. Because students may take both introductory courses in the same term, 
or repeat the same course in multiple terms, it is possible to be assigned to a treatment group 
more than once. Main results use student course enrollment as the unit of analysis, which we 
refer to as the student for brevity. We also check robustness to repeated observations from the 


same student. 


2.3. Data 


We have administrative data and baseline and endline surveys from study participants. 
The baseline and endline surveys were conducted during the first and last two weeks, 
respectively, of each ten-week term. The endline survey therefore occurred in Weeks 9-10, after 
treatment was sent in Week 8. Students earned course credit for completing the surveys. 
Surveys included questions about the likelihood of majoring or minoring in Economics and 


perceptions of the Economics major. Administrative data includes measures of experimental 


2 We (the researchers) sent the messages from instructor accounts. Instructors and the Economics 
Academic Advisor were blinded to the treatment status of individual students. 

3 The experiment also included a second phase, which tested a “resilience” message among a subset 
of better-performing students after the course ended. This paper focuses on Phase One, both for brevity 
and because we failed to find statistically significant effects of the Phase Two intervention. For Phase 
Two description and results, see the appendix and working paper version (Pugatch and Schroeder 2020). 
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take-up, such as whether students opened treatment emails, clicked on links within those 
emails, or scheduled appointments with the Economics Academic Advisor. Administrative data 
also include student demographics, grades, and major. Our main outcome of interest is an 
indicator for majoring in Economics as of Winter 2020.4 Our main results therefore represent 
the effects of the treatments two to four terms after the experiment, ensuring that students 
had sufficient time to reflect on the information and take the necessary administrative steps. 


Other outcomes come from the endline survey. 


3 Methodology 


Our primary specification is the ordinary least squares (OLS) regression: 


4 4 
EconMajor;, = A + ye Ogee >A; (Pras MG) Ae Pee. 11) 
=i gal 


J 


In equation (1), j indexes treatment arms; i indexes students; s indexes strata; EconMajor 


is an indicator for majoring in Economics, the outcome of interest; 7, through 7), are indicators 
for belonging to each of the four treatment arms (basic information, earnings information, 
AEA video, or OSU video; the control group is the omitted category); G is an indicator for 
belonging to a demographic group of interest, such as first generation or underrepresented 
minority; X is a vector of controls, including the main effect of the demographic group Gj the 
baseline outcome, and strata dummies; and « is an error term. The inclusion of strata dummies 
isolates the random variation in treatment status within strata. The baseline outcome adjusts 


for any prior outcome differences between treatment groups and increases precision. We 


4 The data were recorded in January 2020, before COVID-19 cases were widespread in the U.S. We 
did not observe an administrative major in Winter 2020 for 48 students. For these students, the outcome 
is an indicator for being an Economics major in the last term observed, provided this was at least one 
term later than when the student was in the experiment. 
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estimate heteroscedasticity-robust standard errors, consistent with random assignment at the 


student level (Abadie et al. 2017).® 


Our coefficients of interest are B, through B,, which measure differential effects of each 


treatment among students in demographic group G. For instance, 8, measures whether basic 


information changed the proportion of Economics majors from group G differently from the 
effect of this treatment in the rest of the sample. 

We also estimate a simplified version of equation (1) which bundles all treatments into a 
single indicator. We analyze outcomes using the full sample and for the subsample of students 
who earned a B- or above, given past evidence of a greater response to informational nudges 


among better-performing students (Li 2018, Pugatch and Schroeder 2021). 


4 Results 


4.1 Descriptive statistics 


The experiment included 2,277 participants, or 85% of Economics Principles course 
enrollment.’ First generation and underrepresented minority students each comprise 10% of 
the sample, lower than their proportions in the university (23% and 12%, respectively). Table 
1 shows summary statistics for the sample, by first generation and underrepresented minority 
status. Compared to other students, first generation or underrepresented minority students 
are approximately equally likely to be female, but more likely to be white, have lower GPAs, 
and are less likely to expect an A in the course (Panel A, columns 1-2). Overlap among first 
generation and underrepresented minority students is high, with 38% of first generation 
students also URM, and 40% of URM students also first generation. Nonetheless, the imperfect 
overlap suggests they are distinct groups. 

At baseline, the proportion of first generation or URM students majoring in Economics 


was higher than students not from these groups (3% vs. 2%), but not statistically 


> Our companion paper followed an analysis plan focused on gender differences in the response to 
treatment. We build on that analysis plan in this paper, but our focus on different demographic groups 
and differences in specification make the analysis exploratory. 
6 To participate, students had to be at least 18 years old, complete the baseline survey, and consent. 
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distinguishable. However, first generation/URM students were significantly less likely to report 
an intention to minor in Economics, and had lower intentions to major and minor in Economics 
on a 0-100 scale. Perceptions of Economics also differed at baseline, with first generation/URM 
students less likely to cite future income or rewarding careers as the biggest appeal of 
Economics. URM students were more likely to cite lack of diversity as the biggest drawback 
to Economics than non-URM students. 

Table 1, Panel B shows endline outcomes. Endline survey attrition did not differ by first 
generation/URM status. By definition, there is no attrition in the administrative data.’ 
Differences in majoring and minoring in Economics persisted at endline, though these 
differences leave open the possibility that first generation/URM students responded differently 
to treatment. We assess this question in the next subsection. 

Table 2 presents baseline characteristics conditional on treatment status to test for balance. 
Baseline values were imbalanced by treatment status for two characteristics: female and first 
generation. We will present robustness checks which include indicators for these characteristics 


in the controls. 


4.2 Results 


About two-thirds of treated students opened the intervention emails, with similar rates on 
this and other take-up measures between first generation/URM students and students not in 
these groups (Appendix Table Al). In the sample, 71 students majored in Economics by 
Winter 2020, or 3.2%. Of these, 33 students—46% of Economics majors—were not Economics 
majors at baseline. By contrast, only six students switched out of Economics. Of the 33 
students who switched into Economics, six were first generation or underrepresented minority 
students, all of them treated. Of the six students who switched out of Economics, four were 
first generation/URM, with three from the control group. 

The results of estimating equation (1) appear in Table 3. We interact treatment with one 


of three group indicators: 1) first generation or URM, 2) first generation, and 3) URM. Being 


7 The administrative data fails to report post-study major for 39 students, however, because the 
term in which they took Principles is the last term we observe them enrolled at the university. 
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in the combined first-generation or URM group is associated with a lower probability of 


majoring in Economics by four percentage points, significant at 5% (columns 1 (J) 2). The point 


estimates of this coefficient were larger in magnitude, at 10 percentage points, for the 
subsample of students who earned a B- or above in the course, although only weakly 
statistically significant (columns 3-4). 

In these regressions, none of the individual treatments had a statistically significant effect 
on students who were neither first-generation nor URM (columns 1 and 3). The interaction 
terms, however, indicate that there were positive and significant impacts of the treatments for 
students in the first-generation or URM group. The basic information email increased the 
probability that a student in this group majored in Economics by eight percentage points, 
significant at 5% (column 1). This effect was even stronger for the B- and above subsample, 
where the coefficient was 19 percentage points, likewise significant at 5%. In both cases, 
assignment to the basic information treatment more than reversed the gap in majoring in 
Economics for students of this demographic group. The interaction between the AEA video 
treatment and first-generation or URM status produced somewhat smaller and less precise 
coefficients, but still sufficient to overcome the gap, at five percentage points in the full sample 
and 13 percentage points in the B- and above sample (columns 1 and 3). Regressions using an 
indicator for assignment to any treatment email show similar results. The indicator for any 
treatment was insignificant, but the interaction between this indicator and the first-generation 
or URM group was statistically significant at 5%, increasing the probability of majoring in 
Economics by five percentage points in the full sample and by 12 percentage points in the B- 
and above group (columns 2 and 4). 

Allowing for differential effects of the treatments separately for first-generation and URM 
students produces similar results, although less precise. Being a first-generation student was 
associated with a four percentage point lower probability of majoring Economics, significant 
at 10% (columns 5-6); the coefficients were larger in magnitude but insignificant for the B- or 
above group (columns 7-8). The interaction between any treatment and first-generation was 
significant at 5%, increasing the probability of majoring in Economics by five percentage points 


(column 6). For the B- or above group, this coefficient was 13 percentage points, significant 
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at 10% (column 8). Looking at the treatments individually, the coefficients on the interaction 
with the basic information treatment are similar in magnitude to the above estimates, but 
weakly significant (columns 5 and 7). Additionally, interactions between first-generation status 
and the AEA video and OSU video are positive and weakly significant, and the basic 
information email is weakly significant on its own for the B- or above group (columns 5 and 
7). 


The coefficients on the URM indicator have negative signs with slightly smaller magnitudes 


than first generation, but none are statistically significant (columns 9 01 12). Additionally, 


most of the interactions between treatments and URM status are positive, with larger 
magnitudes in the B- or above sample. These results are consistent with the results for first 
generation students, though none of the URM coefficients are statistically different from zero. 

Taken together, these results provide evidence that an email message from a course 
instructor can successfully encourage underrepresented students to major in Economics. The 
positive effects represent a lasting change in revealed preference, since the outcome is measured 


by administrative data collected in the academic year following treatment. Overall, we find 
that a simple nudge—a single email during a 10-week course—can reverse the lower likelihood 


of majoring in Economics found for first-generation or URM students. 


4.3. Robustness checks and mechanisms 

In Appendix Table A2, we check the robustness of our results to several alternative 
specifications. Column (1) adds controls for variables imbalanced at baseline (female and first- 
generation). Column (2) controls for Phase 2 of the experiment, in which two versions of a 
second encouragement message were sent after the course ended to students earning a B- or 
better (see footnote 3). We include dummies for Phase Two control and Phase Two treatment, 
with students not in Phase Two as the omitted category. We do not include Phase Two 
controls in the main specification because eligibility may be affected by the Phase One 
treatment, and Phase Two did not influence the decision to major in Economics (Appendix 
Table A4). We nonetheless check robustness here. Finally, column (3) limits the sample to one 


observation per student, removing duplicate entries of students who took both Principles 
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courses or repeated a course, redefining treatment as number of times exposed to each message, 
and adding an additional control for taking a previous Principles course.? Each specification 
builds on the previous, i.e., column (2) includes the baseline controls from column (1), and 
column (3) includes the baseline and Phase Two controls from columns (1)-(2). 

Results from the first two columns are very similar to the main results from Table 3. In 
column (3), the magnitude and precision of the interaction between treatment and first 
generation or URM status fall. However, the pattern of point estimates remains consistent 
with the main results. A similar patterns repeats for first generation and URM students 
separately in columns (4)-(9), with the first two robustness checks similar to the main results 
and consistent point estimates in the third. Panel B shows similar findings for students earning 
B- or better. The robustness checks demonstrate consistency with the main results.® 

To investigate mechanisms underlying treatment effects, we analyze whether the 
experiment changed student perceptions of Economics, using endline survey data. Student 
views of the biggest appeal (fun to study, future income, or rewarding career) of Economics 
did not respond significantly to any message, either for the main effect or when interacted 
with first-generation/URM (Table 4). There is some evidence that video treatments made 
Economics appear more diverse, with declines in the proportion citing lack of diversity as the 
biggest drawback of the Economics major (main effect of OSU video, and interaction term 
between AEA video and first generation/URM, both significant at 10%). However, the general 
lack of changes in perceptions of Economics in response to messages suggests that targeted 
outreach, regardless of message content, may be sufficient to nudge underrepresented students. 
This interpretation is also consistent with our finding the largest and most precisely estimated 


effects for the basic information message. 


8 Removing duplicates drops 332 instances of a student appearing a second time; 22 instances of a 
student appearing a third time; and one instance of a student appearing a fourth time. Together, these 
dropped observations represent 16% of the original sample size of 2,238. Of the 332 students who appear 
more than once, 236 took both macro and micro without repeating either; 78 took one course and 
repeated it one or more times; and 18 took both courses and repeated one. For students who participated 
in a pilot version of this study in Spring 2018, the number of times they received a treatment includes 
the pilot emails. 

® We also analyze minoring in Economics as an outcome, using self-reported data from the endline 
survey, but fail to find significant treatment effects (Appendix Table A3). 
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4.4 Policy simulation 


Our main results in Table 3 demonstrate a positive effect of informational nudges on 
majoring in Economics among first generation or underrepresented minority students. Suppose 
Oregon State University adopted this intervention as Economics Department policy. How 
would the policy change the prevalence of first generation/URM students among Economics 
majors? 

Table 5 presents results from this counterfactual exercise. The “control” scenario presents 
the status quo, by extrapolating the control group proportion majoring in Economics within 
each student group (columns 2/5) to the entire study population. In this scenario for the full 
sample, 9 first generation/URM students and 41 non-first generation/URM students would be 
Economics majors, for a first generation/URM proportion of 0.18 (column 7). The “treatment” 
scenario adjusts the proportions according to the group-specific point estimates of the 
treatment effect (Table 3, column 2). In this scenario, the implied first generation/URM 
proportion doubles to 0.36. 

Panel B of Table 5 repeats the exercise for the subsample of students earning a B- or 
better. The increase in the first generation/URM proportion under the treatment scenario is 
now even more dramatic, given the greater response to treatment among better-performing 
first generation/URM students (Table32, column 4). The first generation/URM proportion 
rises from 0.14 to 0.35, an increase of 151% over the control scenario. 

A potential objection to this counterfactual exercise is that the absolute numbers of 
students are relatively small. Admittedly, the small base of first generation/URM students 
majoring in Economics render the proportional changes particularly dramatic. Yet the exercise 
is based on relatively precise point estimates—particularly for first generation students— 
generated from an experiment involving more than 2,200 students at a large public university. 
Additionally, this exercise suffers from the well-known limitations of policy simulations in 
partial equilibrium. We also do not account for further changes to the demographic mix of 
Economics students which might occur between the Principles courses and graduation, 


particularly in the absence of other efforts to retain underrepresented students. Our results 
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nevertheless suggest that a simple informational nudge can increase socioeconomic diversity in 


the Economics major. 


5 Conclusion 


Our simple information intervention increased Economics majors among first generation 
students. The contrast between this result and findings from the literature on women in 
Economics highlights the importance of understanding the experiences of different 


underrepresented groups and the factors that may limit interest in Economics. Women have 


earned the majority of bachelorUs degrees in the US for the past 40 years. First-generation 
college students, on the other hand, enter college with less exposure to higher education than 
their peers. A short, informational message about majoring in Economics would be expected 
to have the strongest impact where knowledge gaps are largest. After the gains from an initial 
amount of information are achieved, more intensive work may be required to further increase 
participation in Economics among underrepresented students. 

In this study and Bayer, Bhanot, and Lozano (2019), a one-time message was sufficient to 
change behavior among first generation and URM students. By contrast, successful approaches 
to increase female interest in Economics (e.g., Li 2018; Porter and Serra 2019) included 
personal engagement with students, whereas less successful approaches used only light-touch 
interventions (Bayer, Bhanot, and Lozano 2019; Antman, Flores, and Skoy 2020; Pugatch and 
Schroeder 2021). Together, these results suggest the promise and limitations of nudges to 
increase diversity in Economics. The approaches more likely to succeed address the distinct 
barriers faced by different groups. 

Despite the dramatic results of our experiment among first generation/URM students, we 
acknowledge that light-touch interventions alone are unlikely to sustain increased 
representation of these groups among economists. Within our sample, underrepresented 
minorities cite lack of diversity as the biggest drawback of Economics at more than twice the 
rate of non-URM students. These findings echo the exclusion felt by underrepresented students 
in Economics documented elsewhere (Bayer, Bhanot, et al. 2020; Bayer, Hoover, and 


Washington 2020). Targeted support programs to students underrepresented in the discipline 
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offer promise but remain relatively rare, with the most effective program elements still 
unknown. Shifting the content of Principles courses (Bayer, Bruich, et al. 2020; Benjamin, 
Cohen, and Hamilton 2020; Bowles and Carlin 2020; Owen and Hagstrom 2021) may also 
complement targeted messaging and engagement to increase diversity in Economics. Future 
work may determine whether these changes complement or substitute for marketing efforts to 
diversify the student population. Pursuing these questions will help to better understand how 


to promote diversity in Economics. 
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7 Tables 


Table 1: Summary statistics 


1st gen or URM 1st generation URM 
no yes no yes no yes 
(1) (2) (3) (4) (5) (6) 

Panel A: baseline characteristics 
female 0.36 0.33 0.35 0.35 0.36 0.31 
white 0.65 0.24*** 0.61 0.37*** 0.65 0.00*** 
First generation 0.00 0.64*** 0.00 1.00 0.07 0.40*** 
underrepresented minority 0.00 0.6*** 0.07 0.38*** 0.00 1.00 
underrepresented minority (inc. multiple race) 0.07 0.64*** 0.13 0.44*** 0.07 1.00*** 
high school GPA 3.49 3.40*** 3.48 3.41** 3.48 3.37*** 
GPA at Oregon State, previous terms 3.09 2.94*** 3.08 2.95*** 3.08 2.92*** 
Expected grade: A 0.51 0.46* 0.51 0.44* 0.51 0.45* 
Expected grade: B 0.44 0.46 0.44 0.48 0.44 0.48 
Economics major 0.02 0.03 0.02 0.03 0.02 0.02 
Intends to minor in Economics (0/1) 0.05 0.02*** 0.05 0.02** 0.05 0.00*** 
Intends to major in Economics (0-100) 19.1 15.0*** 18.9 14.5** 18.9 14.2*** 
Intends to minor in Economics (0-100) 27.5 21.6*** 27.2 20.7*** 27.0 22.4** 
Biggest appeal of Economics major: 

fun to study 0.18 0.20 0.18 0.19 0.18 0.20 

income 0.37 0.32* 0.37 0.31* 0.37 0.31* 

rewarding career 0.21 0.17** 0.21 0.17 0.21 0.18 
Biggest drawback of Economics major: 

boring 0.23 0.21 0.23 0.18** 0.23 0.20 

too difficult 0.38 0.34 0.38 0.37 0.38 0.34 

too focused on making money 0.07 0.06 0.07 0.06 0.07 0.06 

lack of diversity 0.03 0.05 0.04 0.03 0.03 0.08** 
Panel B: outcomes 
completed endline survey 0.87 0.85 0.87 0.86 0.87 0.85 
course grade (0-4) 2.40 2.17*** 2.39 2.17*** 2.40 2.08*** 
Economics major 0.03 0.04 0.03 0.04 0.03 0.03 
Intends to minor in Economics (0/1) 0.04 0.02** 0.04 0.02 0.04 0.02** 
Intends to major in Economics (0-100) 18.5 15.2** 18.2 15.7 18.4 14.3** 
Intends to minor in Economics (0-100) 26.2 21.9** 25.8 22.8 25.9 21.6** 
Biggest appeal of Economics major: 

fun to study 0.15 0.14 0.15 0.14 0.15 0.13 

income 0.40 0.33** 0.40 0.31** 0.39 0.34 

rewarding career 0.18 0.19 0.18 0.19 0.18 0.20 
Biggest drawback of Economics major: 

boring 0.26 0.26 0.26 0.25 0.26 0.25 

too difficult 0.40 0.38 0.40 0.40 0.40 0.38 

too focused on making money 0.06 0.06 0.06 0.06 0.06 0.06 

lack of diversity 0.04 0.06 0.04 0.06 0.04 0.09** 
took Economics course (after Principles) 0.03 0.06* 0.03 0.05 0.03 0.06 
N 1,908 369 2,041 236 2,054 223 
Proportion 0.84 0.16 0.90 0.10 0.90 0.10 


Table shows baseline characteristics and outcomes by first generation and underrepresented minority (URM) status. 


Sample is all students who participated in study. Underrepresented minority is defined as American Indian or Alaska 
Native, Balck or African American, Hispanic, or Native Hawaiian or Pacific Islander. URM status does not include 
multiracial or international students. Data sources: administrative data and baseline and endline surveys. Stars 
indicate significant differences with column to left. * significant at 10%, ** significant at 5%, *** significant at 1%. 


19 


Table 2: Baseline balance 


Treatment arm 
control Basic AEA earnings OSU F-test 
information video information video (p-value) 
(1) (2) (3) (4) (5) (6) 

female 0.32 0.41 0.36 0.35 0.33 0.05 
[0.47] [0.49] [0.48] [0.48] [0.47] 

white 0.57 0.58 0.59 0.58 0.61 0.84 
[0.49] [0.49] [0.49] [0.49] [0.49] 

Asian 0.08 0.08 0.06 0.06 0.07 0.67 
[0.27] [0.27] [0.24] [0.24] [0.26] 

Underrepresented minority 0.10 0.09 0.10 0.11 0.08 0.51 
[0.30] [0.29] [0.30] [0.32] [0.27] 

1st generation 0.14 0.08 0.09 0.09 0.12 0.03 
[0.35] [0.27] [0.29] [0.29] [0.32] 

High school GPA 3.46 3.50 3.45 3.49 3.46 0.20 
[0.40] [0.37] [0.42] [0.37] [0.42] 

Oregon State GPA 3.05 3.05 3.07 3.10 3.05 0.52 
[0.56] [0.56] [0.56] [0.52] [0.58] 

expected grade: A 0.48 0.49 0.48 0.54 0.52 0.22 
[0.50] [0.50] [0.50] [0.50] [0.50] 

expected grade: B 0.46 0.45 0.47 0.41 0.42 0.22 
[0.50] [0.50] [0.50] [0.49] [0.49] 

intends to major in Economics 0.03 0.02 0.03 0.04 0.05 0.18 
[0.17] [0.14] [0.18] [0.19] [0.21] 

intends to minor in Economics 0.05 0.03 0.04 0.06 0.03 0.28 
[0.22] [0.18] [0.20] [0.24] [0.18] 

likelinood of majoring in Economics (0-100) | 17.45 17.25 19.16 20.12 18.07 0.38 
[24.40] [24.34] [26.13] [26.09] [25.27] 

likelinood of minoring in Economics (0-100) | 26.23 25.68 27.18 27.37 26.22 0.87 
[26.91] [27.18] (27.11] [28.03] [25.85] 

completed endline survey 0.87 0.85 0.86 0.89 0.87 0.51 
[0.34] [0.36] [0.35] [0.31] [0.34] 

N 456 455 455 460 451 2,277 


Table shows mean of baseline characteristics, by study arm. Standard deviations in brackets. Column (6) report p-values 
of joint test of treatment dummies on baseline characteristic, controlling for strata dummies. Underrepresented minority 
is defined as American Indian or Alaska Native, Balck or African American, Hispanic, or Native Hawaiian or Pacific 
Islander. URM status does not include multiracial or international students. 
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Table 3: Results 


Outcome Major in Economics 
Group First generation or URM First generation URM 
Sample All B- or above all B- or above all B- or above 
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) 
basic information 0.006 0.019 0.011 0.029* 0.015 0.038** 
(0.009) (0.016) (0.009) (0.017) (0.009) (0.018) 
earnings information 0.011 0.025 0.012 0.027 0.015 0.033* 
(0.010) (0.020) (0.010) (0.019) (0.010) (0.019) 
AEA video 0.003 0.010 0.004 0.012 0.008 0.020 
(0.008) (0.016) (0.008) (0.017) (0.008) (0.016) 
OSU video -0.003 -0.004 -0.002 -0.003 0.005 0.010 
(0.008) (0.014) (0.008) (0.015) (0.008) (0.015) 
basic info*group 0.081** 0.188** 0.080* 0.188* 0.045 0.100 
(0.033) (0.083) (0.046) (0.114) (0.037) (0.104) 
earnings*group 0.023 0.064 0.021 0.052 0.005 0.030 
(0.020) (0.058) (0.021) (0.059) (0.023) (0.082) 
AEA video*group 0.048* 0.128* 0.059* 0.155 0.033 0.110 
(0.025) (0.075) (0.034) (0.097) (0.031) (0.115) 
OSU video*group 0.041 0.108 0.056* 0.134* -0.006 -0.022 
(0.027) (0.073) (0.029) (0.077) (0.032) (0.117) 
group -0.043** -0.043** -0.104* -0.102* | -0.038* -0.038* -0.081 -0.080 | -0.024 -0.024  -0.072 -0.069 
(0.019) (0.019) (0.054) (0.054) | (0.021) (0.021) (0.058) (0.058) | (0.020) (0.020) (0.078) (0.078) 
treatment 0.004 0.013 0.006 0.017 0.011 0.025** 
(0.007) (0.012) (0.007) (0.012) (0.007) (0.012) 
treatment*group 0.048** 0.120** 0.052** 0.126* 0.020 0.061 
(0.021) (0.059) (0.024) (0.065) (0.024) (0.085) 
N 2,238 2,238 1,003 1,003 2,238 2,238 1,003 1,003 2,238 2,238 1,003 1,003 
Control mean 0.023 0.023 0.037 0.037 0.023 0.023 0.037 0.037 0.023 0.023 0.037 0.037 
all interactions=0 0.09 0.02 0.15 0.04 0.11 0.03 0.16 0.05 0.52 0.40 0.64 0.47 


Sample is all students who consented to participate in study. Outcome is dummy for majoring in Economics, from administrative data in Winter 2020 or most 
recent available. Underrepresented minority (URM) is defined as American Indian or Alaska Native, Balck or African American, Hispanic, or Native Hawaiian 
or Pacific Islander. URM status does not include multiracial or international students. All regressions include strata dummies and control for baseline 
outcome. Heteroskedasticity-robust standard errors in parenthesis. * significant at 10%, ** significant at 5%, *** significant at 1% 
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Table 4: Perceptions of Economics, first-generation or URM 


Biggest appeal of Economics major Biggest drawback of Economics major 
fun to study future income rewarding career | focused on money boring lack of diversity 
basic information -0.009 0.001 -0.023 -0.011 0.013 -0.017 
(0.026) (0.036) (0.029) (0.018) (0.031) (0.015) 
earnings information -0.006 0.025 -0.003 -0.024 0.018 -0.001 
(0.025) (0.036) (0.029) (0.017) (0.032) (0.016) 
AEA video -0.025 0.054 -0.043 -0.016 0.050 -0.002 
(0.025) (0.037) (0.029) (0.017) (0.032) (0.016) 
OSU video -0.019 0.027 -0.002 0.027 0.068** -0.026* 
(0.025) (0.036) (0.030) (0.020) (0.031) (0.014) 
basic info*1‘t generation or URM -0.017 0.127 -0.031 -0.015 -0.068 -0.003 
(0.059) (0.084) (0.070) (0.041) (0.080) (0.045) 
earnings*1°t generation or URM 0.052 0.052 -0.019 0.018 -0.062 0.018 
(0.062) (0.084) (0.073) (0.045) (0.079) (0.050) 
AEA video*1* generation or URM | -0.005 0.002 0.009 0.000 0.080 -0.072* 
(0.060) (0.085) (0.071) (0.042) (0.084) (0.039) 
OSU video*1* generation or URM | 0.038 -0.043 0.036 -0.002 -0.053 0.020 
(0.058) (0.082) (0.073) (0.050) (0.076) (0.047) 
group -0.023 -0.023 -0.077 -0.078 0.028 0.028 0.004 0.004 0.038 0.037 0.025 0.025 
(0.038) (0.038) (0.055) (0.055) (0.050) (0.049) | (0.030) (0.030) (0.054) (0.054) (0.031) (0.031) 
treatment -0.015 0.027 -0.018 -0.006 0.037 -0.011 
(0.020) (0.029) (0.024) (0.014) (0.025) (0.012) 
treatment* 1° generation or URM 0.017 0.034 -0.001 0.001 -0.024 -0.009 
(0.045) (0.064) (0.056) (0.034) (0.062) (0.036) 
N 1,976 1,976 1,976 1,976 1,976 1,976 1,976 1,976 1,976 1,976 1,976 1,976 
control mean 0.159 0.159 0.361 0.361 0.189 0.189 0.061 0.061 0.232 0.232 0.053 0.053 
all interactions == 0.796 0.704 0.356 0.591 0.909 0.980 0.965 0.980 0.398 0.696 0.0688 0.792 


Table reports coefficients of regressions of indicated outcome on treatment status. Outcomes are responses to endline survey questions on biggest 
appeal/drawback of Economics major. Sample is all study participants who completed endline survey. All regressions include strata dummies and 
baseline outcome. Heteroskedasticity-robust standard errors in parentheses. * significant at 10%, ** significant at 5%, *** significant at 1%. 
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Table 5: Economics majors, counterfactual exercises 


not first generation or URM first generation or URM projected 
Base major base major first-gen/URM 
population proportion projected | population proportion projected proportion 
scenario (1) (2) (3) (4) (5) (6) (7) 
Panel A: all students 
control 1,882 0.022 41 356 0.025 9 0.18 
treatment 1,882 0.026 49 356 0.077 28 0.36 
Panel B: B- or better 
control 874 0.036 31 129 0.042 5 0.14 
treatment 874 0.049 43 129 0.175 23 0.35 


Table shows projected proportions and numbers of Economics majors under scenarios listed in first column. “Control” scenario based on proportions 


majoring in Economics among control group. “Treatment” scenario based on change in proportion majoring in Economics in response to treatment. 
Base population refers to sample size within the study population. Column (7) shows projected first generation or URM student proportion among 
Economics majors, i.e., column (6) /(column (6)+column(3)). 


23 


A Simple Nudge Increases Socioeconomic Diversity in Undergraduate Economics: Appendix 


8 Appendix: Intervention messages 


Figure Al(a): Phase One: basic information 


ECON 201: Consider majoring in Economics! 
Jon Chesbro <jon.chesbro@oregonstate.edu> 

Mon 5/20/2019 12:00 PM 

To: Schroeder, Elizabeth <Liz Schroeder@oregonstate.edu> 


Having Youble reading ts? To view this enal as 8 web page, dick hare 
Hi Liz, 


| hope you have enjoyed learning about Economics this term. As you plan your 
future studies, | encourage you to consider majoring or minoring in Economics. 
In addition to a traditional Economics degree, Oregon State University's Economics 
Program offers options in Managerial Economics; Law, Economics and Policy; and 
Mathematical Economics. Economics training provides excellent preparation for 
graduate work in Economics, Public Policy, Law, and Business. OSU Economics 
graduates also use their degrees to work professionally in the public (federal, state 
and local government) and private (banking, consulting, retail, and corporate) 
sectors. 


If you are interested in majoring or minoring in Economics or would like to learn 
more, please make an appointment with Laura Relyea, the Economics Academic 
Advisor. 


Sincerely, 
Jon Chesbro 
Instructor, Economics 


You may update your profile here 
This email was sent by: Oregon State University, Printing & Mailing Services,4700 SW Research Way, Corvallis, OR 97333 
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A Simple Nudge Increases Socioeconomic Diversity in Undergraduate Economics: Appendix 


Figure Al(b): Phase One: earnings information 


ECON 201: Consider majoring in Economics! 
Camille Nelson <camille.nelson@oregonstate.edu> 

Mon 5/20/2019 12:00 PM 

To: Schroeder, Elizabeth <Liz.Schroeder@ oregonstate.edu> 


Having trovtile reading this? To view this emai as 3 web page, click here 
Hi Liz, 


| hope you have enjoyed learning about Economics this term. As you plan your future 
studies, | encourage you to consider majoring or minoring in Economics. In addition to 
a traditional Economics degree, Oregon State University’s Economics Program offers 
options in Managerial Economics; Law, Economics and Policy; and Mathematical 
Economics. Economics training provides excellent preparation for graduate work in 
Economics, Public Policy, Law, and Business. OSU Economics graduates also use their 
degrees to work professionally in the public (federal, state and local government) and private 
(banking, consulting, retail, and corporate) sectors. 


Majoring in Economics can be a smart career decision. Average earnings for economics 


majors are higher than the overall average for college graduates, both at the start of their 
careers and throughout their lives. 


Salary by college major 


aii) 


‘250,000 


80,000 1090" 


80,020 
1 


Medan sela’y 


20,000 40,000 


G 


1 year afer graduation 15 years after graduation 
GR Econ majors «© ERE: AN college graduates 


Source: hamiltonproject.org/charts/career_earnings_by_college_major/ 


If you are interested in majoring or minoring in Economics or would like to learn more, 
please make an appointment with Laura Relyea, the Economics Academic Advisor. 


Sincerely, 
Camille Nelson 
Senior Instructor, Economics 


You may update your profile here 
This emai was sent by: Oragon State University, Printing & Mailing Services,4700 SW Research Way, Corvallis, OR 97333 
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A Simple Nudge Increases Socioeconomic Diversity in Undergraduate Economics: Appendix 


Figure Al(c): Phase One: AEA video 


ECON 202: Consider majoring in Economics! 
Mike Nelson <mike.nelson@oregonstate.edu> 

Mon 5/20/2019 12:00 PM 

To: Schroeder, Elizabeth <Liz.Schroeder@oregonstate.edu> 


Having trouble reading this? To view this email as a web page, dick here. 
Hi Liz, 


| hope you have enjoyed learning about Economics this term. As you plan your future 
studies, | encourage you to consider majoring or minoring in Economics. In addition to 
a traditional Economics degree, Oregon State University's Economics Program offers 
options in Managerial Economics; Law, Economics and Policy; and Mathematical 
Economics. Economics training provides excellent preparation for graduate work in 
Economics, Public Policy, Law, and Business. OSU Economics graduates also use their 
degrees to work professionally in the public (federal, state and local government) and private 
(banking, consulting, retail, and corporate) sectors. 


Learn more in this video about careers in Economics...it's much more than you think! 


If you are interested in majoring or minoring in Economics or would like to learn 
more, please make an appointment with Laura Relyea, the Economics Academic 
Advisor. 


Sincerely, 
Mike Nelson 
Senior Instructor, Economics 


You may update your profile hare. 
This email was sent by: Oregon State University, Printing & Mailing Services.4700 SW Research Way, Corvallis, OR 97333 
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A Simple Nudge Increases Socioeconomic Diversity in Undergraduate Economics: Appendix 


Figure Al(d): Phase One: OSU video 


ECON 201: Consider majoring in Economics! 
Camille Nelson <camille.nelson@oregonstate.edu> 

Mon 5/20/2019 12:00 PM 

To: Schroeder, Elizabeth <Liz.Schroeder@oregonstate.edu> 


Having trouble reading this? To view this email as a web page, click here. 
Hi Liz, 


| hope you have enjoyed learning about Economics this term. As you plan your future 
studies, | encourage you to consider majoring or minoring in Economics. In addition to 
a traditional Economics degree, Oregon State University's Economics Program offers 
options in Managerial Economics; Law, Economics and Policy; and Mathematical 
Economics. Economics training provides excellent preparation for graduate work in 
Economics, Public Policy, Law, and Business. OSU Economics graduates also use their 
degrees to work professionally in the public (federal, state and local government) and private 
(banking, consulting, retail, and corporate) sectors. 


Learn more in this video about what it's like to major in Economics at Oregon State! 


- 


' 


-_ 
—— 


- 


ALEXANDRA GIZA 
ECONOMICS MAJOR 
GRADUATE 


If you are interested in majoring or minoring in Economics or would like to learn 
more, please make an appointment with Laura Relyea, the Economics Academic 
Advisor. 


Sincerely, 
Camille Nelson 
Senior Instructor, Economics 


You may update your profile here. 
This email was sent by: Oregon State University, Printing & Mailing Services.4700 SW Research Way, Corvallis, OR 97333 
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9 Appendix tables 


Table Al: Take-up and knowledge 


not 1st generation or URM 1st generation or URM 
control placebo earnings AEA OSU F-test control placebo earnings AEA OSU F-test 
email information video video (p-value) email information video video (p-value) 
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) 
opened encouragement email 0.00 0.65 0.73 0.69 0.65 0.00 0.00 0.63 0.63 0.62 0.74 0.00 
[0.00] [0.48] [0.45] [0.46] [0.48] [0.00] [0.49] [0.49] [0.49] [0.44] 
clicked link in email 0.00 0.00 0.02 0.00 0.01 0.02 0.00 0.01 0.00 0.00 0.00 0.92 
[0.00] [0.00] [0.14] [0.00] [0.10] [0.00] [0.12] [0.00] [0.00] [0.00] 
made appointment with Economics advisor 0.00 0.01 0.01 0.01 0.01 0.27 0.00 0.03 0.00 0.00 0.00 0.76 
[0.05] [0.09] [0.11] [0.11] [0.10] [0.00] [0.17] [0.00] [0.00] [0.00] 
viewed AEA video (self-report) 0.16 0.19 0.20 0.23 0.22 0.10 0.16 0.20 0.26 0.23 0.24 0.75 
[0.37] [0.39] [0.40] [0.42] [0.42] [0.37] [0.40] [0.44] [0.42] [0.43] 
viewed OSU video (self-report) 0.16 0.19 0.21 0.24 0.21 0.22 0.18 0.23 0.28 0.21 0.26 0.87 
[0.37] [0.39] [0.41] [0.43] [0.40] [0.39] [0.43] [0.45] [0.41] [0.44] 
1st year salary range correct 0.22 0.21 0.20 0.24 0.27 0.17 0.28 0.23 0.28 0.29 0.29 0.92 
[0.41] [0.41] [0.40] [0.43] [0.44] [0.45] [0.43] [0.45] [0.46] [0.46] 
15th year salary range correct 0.17 0.25 0.21 0.22 0.19 0.14 0.19 0.13 0.21 0.24 0.29 0.35 
[0.37] [0.43] [0.41] [0.42] [0.39] [0.40] [0. = [0.41] [0.43]  [0. = 
372 385 389 384 378 84 71 71 


Table I > shows mean of listed characteristic in each treatment arm (standard deviations in brackets). Sample is all students who participated in stuc Sample is aT students who PRET in study. 


Underrepresented minority is defined as American Indian or Alaska Native, Balck or African American, Hispanic, or Native Hawaiian or Pacific 
Islander. URM status does not include multiracial or international students. Columns (6) & (12) report p-value of F-test of joint hypothesis that all 
treatment arms predict characteristic. p-values adjust for stratification of treatment 
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Table A2: Major in Economics, robustness checks 


Major in Economics 


Group First generation or URM First generation URM 
(1) (2) (3) (4) (5) (6) (7) (8) (9) 
Panel A: full sample 
group -0.056** -0.051** -0.034** -0.039* -0.035 -0.017 -0.026 -0.020 -0.011 
(0.023) (0.024) (0.016) (0.021) (0.022) (0.020) (0.022) (0.023) (0.013) 
treatment 0.004 0.005 0.002 0.006 0.007 0.003 0.011 0.011* 0.005 
(0.007) (0.006) (0.004) (0.007) (0.007) (0.004) (0.007) (0.007) (0.004) 
treatment*group 0.051** 0.047** 0.024 | 0.053**  0.050** 0.024 0.020 0.015 0.006 
(0.021) (0.021) (0.015) (0.024) (0.024) (0.018) (0.024) (0.024) (0.015) 
N 2,238 2,238 1,883 2,238 2,238 1,883 2,238 2,238 1,883 
control mean 0.023 0.023 0.010 0.023 0.023 0.010 0.023 0.023 0.010 
Panel B: B- or above 
group -0.156** -0.155** -0.091** -0.082 -0.082 -0.025 -0.079 -0.079 -0.033 
(0.066) (0.066) (0.043) (0.057) (0.057) (0.047) (0.080) (0.079) (0.039) 
treatment 0.013 0.013 0.005 0.017 0.017 0.007 | 0.026**  0.026** 0.011 
(0.012) (0.011) (0.008) (0.012) (0.012) (0.008) (0.012) (0.012) (0.009) 
treatment*group 0.135** 0.134** 0.056 0.127* 0.126* 0.044 0.066 0.065 0.016 
(0.060) (0.060) (0.039) (0.065) (0.065) (0.042) (0.085) (0.084) (0.043) 
N 1,003 1,003 864 1,003 1,003 864 1,003 1,003 864 
control mean 0.037 0.037 0.014 0.037 0.037 0.014 0.037 0.037 0.014 
control for baseline imbalance X X Xx X X X 
control for Phase 2 X X X 
intensity of treatment x x x 


Sample is all students who consented to participate in study. Outcome is dummy for majoring in Economics, from administrative data in 
Winter 2020 or most recent available. All regressions include strata dummies and baseline outcome. Controls for variables imbalanced at 


baseline are first generation student dummy and female dummy. Controls for Phase 2 are dummies for Phase 2 control and Phase 2 


treatment (omitted category is not in Phase 2). Specifications for “intensity of treatment 
keep only last term observed. Explanatory variables in intensity of treatment specifications represent number of times exposed to 


remove duplicate observations of student and 


treatment, with additional control for taking a previous course. Heteroskedasticity-robust standard errors in parenthesis. * significant at 
10%, ** significant at 5%, *** significant at 1% 
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Table A3: Minor in Economics 


Outcome Minor in Economics 
Group First generation or URM First generation URM 
Sample all B- or above all B- or above all B- or above 
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) 
basic information -0.001 0.001 0.001 0.007 0.001 -0.000 
(0.012) (0.019) (0.012) (0.019) (0.012) (0.018) 
earnings information 0.023 0.031 0.021 0.029 0.023* 0.030 
(0.015) (0.020) (0.014) (0.020) (0.014) (0.019) 
AEA video 0.002 -0.009 0.004 -0.005 0.001 -0.009 
(0.013) (0.017) (0.012) (0.017) (0.012) (0.016) 
OSU video -0.006 -0.012 -0.006 -0.012 -0.002 -0.004 
(0.011) (0.014) (0.011) (0.014) (0.011) (0.015) 
basic info*group 0.016 0.031 0.004 -0.030 0.002 0.069 
(0.031) (0.044) (0.045) (0.045) (0.038) (0.056) 
earnings* group -0.030 -0.026 -0.029 -0.023 -0.041 -0.017 
(0.022) (0.022) (0.026) (0.024) (0.030) (0.023) 
AEA video*group -0.013 0.041 -0.034 0.009 -0.007 0.076 
(0.024) (0.031) (0.026) (0.020) (0.033) (0.052) 
OSU video*group 0.016 0.070 0.020 0.097 -0.024 0.006 
(0.027) (0.045) (0.034) (0.060) (0.029) (0.026) 
group 0.000 0.001 -0.012 -0.011 0.004 0.004 -0.013 -0.012 0.013 0.013 -0.018 -0.016 
(0.017) (0.017) (0.013) (0.013) (0.022) (0.021) (0.014) (0.014) (0.028) (0.028) (0.019) (0.019) 
treatment 0.005 0.004 0.005 0.006 0.006 0.005 
(0.010) (0.013) (0.010) (0.013) (0.009) (0.013) 
treatment*group -0.003 0.028 -0.010 0.018 -0.017 0.035 
(0.020) (0.021) (0.025) (0.025) (0.029) (0.028) 
N 1,976 1,976 953 953 1,976 1,976 953 953 1,976 1,976 953 953 
Control mean 0.0328 0.0328 0.0270 0.0270 0.0328 0.0328 0.0270 0.0270 0.0328 0.0328 0.0270 0.0270 
all interactions=0 0.248 0.872 0.157 0.199 0.312 0.695 0.301 0.484 0.324 0.569 0.252 0.213 


Table reports coefficients of regressions of self-reported intention to minor from endline survey in Economics on treatment status. Sample is all students who consented to 
participate in study. All regressions include strata dummies and baseline outcome. Robust standard errors in parenthesis. * significant at 10%, ** significant at 5%, *** 
significant at 1% 


30 


A Simple Nudge Increases Socioeconomic Diversity in Undergraduate Economics: Appendix 


10 Appendix: Phase Two description and results 


In Phase Two, students who participated in Phase One and earned a grade of at least B- or 
above were enrolled in the study. We randomly assigned these students to receive one of two emails: 


1. Control message of congratulations on their course performance, with encouragement 
to major in Economics. 


2. Treatment: control message content, plus “resilience” message acknowledging that 
Economics can be difficult and encouraging adoption of a growth mindset. 


Phase Two messages were sent once, at the beginning of the academic term following the Phase 
One course, from the email account of the Economics Academic Advisor. Both messages had the 


subject, “ECON invite.” Figure A2 shows each message. Treatment assignment stratified by course 
section, class year (freshman/sophomore/other), and course grade. 
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Figure A2(a): Phase Two: control 


ECON invite 


Laura Relyea <laura.relyea@oregonstate.edu> 
Tue 1/8/2019 12:01 PM 


To; Schroeder, Elizabeth <Liz, Schroeder@oregonstate.edu> 


Haring trouble reading this? To view this email as a web page. click hare. 


Hi Liz, 


Your principles of economics instructor indicated that you performed well in a recent 
class-congratulations! As a result, the economics faculty would like to encourage you to major 
in Economics. 


We have four choices for you: the regular economics major, and options in managerial 
economics, mathematical economics, and law, economics and policy. We also offer an 
economics minor to complement your current major. 


Why Economics? 


A Career in Economics...it's much more than you think. 

¢ Look at what economists do. 

« OSU economics graduates go on to: 

o public sector: federal, state, and local government 

© private sector: banking, consulting, retail, and corporate sectors. 
© advanced study: economics, public policy, law, and business. 


« Career eamings - what economists can expect to earn in different fields 


Please make an appointment with me if you would like to learn more about our program. To set 
up a conversation, click here or on the yellow "schedule your appointment" post-it-note below. | 
look forward to sharing the opportunities in economics with you! 


Best, 
Laura 


Laura Relyea, Academic Advisor 

School of Public Policy | Economics Program 
Oregon State University | Bexell Hall, 418E 
Laura.Relyea@oregonstate.edu 

P: (541) 737-2369 | F: (541) 737-2289 
http://liberalarts.oregonstate.edu/spp/econ/ 


Students please include your name and OSU ID# on all inquiries. 


ey feo 


You may update your profile here 
This email was sent by: Oregon State University, Printing & Mailing Services,4700 SW Research Way, Corvallis, OR 97333 


Events Calendar 


32 


A Simple Nudge Increases Socioeconomic Diversity in Undergraduate Economics: Appendix 


Figure A2(b): Phase Two: treatment 


ECON invite 


Laura Relyea <laura.relyea@oregonstate.edu> 
Tue 1/8/2019 12:00 PM 


To: Schroeder, Elizabeth Liz Schroeder@oregonstate.edu 


Having foutle reading this? To view this email os 3 web page, cick here 
Hi Liz, 


Your principles of economics instructor indicated that you performed well in a recent 
class-congratulations! As a result, the economics faculty would like to encourage you to major 


in Economics. 


Even if your grade was not as high as you had hoped, we encourage you to take additional 
Economics courses. Economics can be a challenging subject. Your Economics ability can grow 
with dedication and hard work. Adopting this "growth mindset" can help you continue to improve 
and succeed as you take more advanced courses. 


We have four choices for you: the regular economics major, and options in managerial 
economics, mathematical economics, and law, economics and policy. We also offer an 
economics minor to complement your current major. 


Why Economics? 


e A Career in Economics...it's much more than you think. 

« Look at what economists do. 

« OSU economics graduates go on to: 
e public sector: federal, state, and local government 
© private sector: banking, consulting, retail, and corporate sectors. 
oe advanced study: economics, public policy, law, and business. 


e Career earings - what economists can expect to earn in different fields 


Please make an appointment with me if you would like to learn more about our program. To set 
up a conversation, click here or on the yellow "schedule your appointment” post-it-note below. | 
look forward to sharing the opportunities in economics with you! 


Best, 
Laura 


Laura Relyea, Academic Advisor 

School of Public Policy | Economics Program 
Oregon State University | Bexell Hall, 418E 
Laura, Relyea@oregonstate.edu 

P: (541) 737-2369 | F: (541) 737-2289 


Students please include your name and OSU ID# on all inquiries. 


Sh 
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Table A4: Phase Two 
Outcome: major in Economics 


sample all Astudents Bstudents disappointed 
(1) (2) (3) (4) 

first generation or URM -0.009 -0.010 -0.035 -0.021 
(0.032) (0.052) (0.027) (0.029) 

Phase Two treatment 0.000 -0.015 0.005 0.005 
(0.012) (0.022) (0.012) (0.017) 

Phase Two treatment * first generation/URM 0.003 0.024 0.030 -0.032 
(0.053) (0.123) (0.046) (0.047) 

N 974 368 600 354 

control mean 0.06 0.10 0.03 0.04 


Sample is students receiving B- or above in Economics Principles. Outcome is majoring in 
Economics from administrative data. Sample in column (2) is students receiving at least A- in 
Economics Principles. Sample in column (3) is students receiving B+, B, or B- in Economics 
Principles. Sample in Panel D is ”disappointed” students, defined as students who expected an A 
grade in course but received a B+ or below. All regressions control for Phae One treatment. 
Robust standard errors in parenthesis. * significant at 10%, ** significant at 5%, *** significant 


at 1% 


34 


